As compared to conventional cathode-ray tubes (CRTs) primarily used for realizing moving images, LCDs (Liquid Crystal Displays) have a drawback, so-called motion blur, which is the blurring of outline of a moving portion perceived by a viewer when displaying an image with movement. It is indicated that this motion blur arises from the LCD display system itself (see e.g., Specification of Japanese Patent No. 3295437; “Hidekazu Ishiguro and Taiichiro Kurita, “Consideration on Motion Picture Quality of the Hold Type Display with an octuple-rate CRT”, IEICE Technical Report, Institute of Electronics, Information and Communication Engineers, EID96-4 (1996-06), p. 19-26”).
Since fluorescent material is scanned by an electron beam to cause emission of light for display in CRTs, the light emission of each pixel is basically impulse-like although slight afterglow of the fluorescent material exists. This is called an impulse-type display system. On the other hand, in the case of LCDs, an electric charge accumulated by applying an electric field to liquid crystal is retained at a relatively high rate until the next electric field is applied. Especially, in the case of the TFT system, since a TFT switch is disposed for each dot configuring a pixel and usually each pixel is provided with an auxiliary capacity, the ability to retain the accumulated charge is extremely high. Therefore, the light emission is continued until the pixels are rewritten by the application of the electric field based on the image information of the next frame or field (hereinafter, represented by the frame). This is called a holding-type display system.
Since the impulse response of the image displaying light has a temporal spread in the above holding-type display system, special frequency characteristics deteriorate as temporal frequency characteristics deteriorate, and the motion blur occurs. Since the human eye can smoothly follow a moving object, if the light emission time is long as in the case of the holding type, the movement of image looks jerky and unnatural due to the time integration effects.
To improve the motion blur in the above holding-type display system, a technique for converting a frame rate (number of frames) by interpolating an image between frames is known. This technique is called FRC (Frame Rate Converter) and is put to practical use in liquid crystal display devices, etc.
Conventionally known methods of converting the frame rate include various techniques such as simply repeating read-out of the same frame for a plurality of times and frame interpolation using linear interpolation between frames (see e.g., Tatsuro Yamauchi, “TV Standards Conversion”, Journal of the Institute of Television Engineers of Japan, Vol. 45, No. 12, pp. 1534-1543 (1991)). However, in the case of the frame interpolation processing using the linear interpolation, unnaturalness of motion (jerkiness, judder) is generated due to the frame rate conversion, and the motion blur disturbance due to the holding-type display system cannot sufficiently be improved, resulting in inadequate image quality.
To eliminate effects of the jerkiness, etc., and improve quality of moving images, a motion-compensated frame interpolation processing using motion vectors has been proposed. Since a moving image itself is captured to compensate the image movement in this process, highly natural moving images may be acquired without deteriorating the resolution and generating the jerkiness. Since interpolation image signals are generated with motion compensation, the motion blur disturbance due to the above holding-type display system may sufficiently be improved.
Above Specification of Japanese Patent No. 3295437 discloses a technology of motion-adaptively generating interpolation frames to increase a frame frequency of a displayed image for improving deterioration of spatial frequency characteristics causing the motion blur. In this case, at least one interpolation image signal interpolated between frames of a displayed image is motion-adaptively created from the previous and subsequent frames, and the created interpolation image signals are interpolated between the frames and are sequentially displayed.
FIG. 1 is a block diagram of a schematic configuration of an FRC drive display circuit in a conventional liquid crystal display device and, in FIG. 1, the FRC drive display circuit includes an FRC portion 100 that converts the number of frames of the input image signal by interpolating the image signals subjected to the motion compensation processing between frames of the input video signal, an active-matrix liquid crystal display panel 203 having a liquid crystal layer and an electrode for applying the scan signal and the data signal to the liquid crystal layer, and an electrode driving portion 204 for driving a scan electrode and a data electrode of the liquid crystal display panel 203 based on the image signal subjected to the frame rate conversion by the FRC portion 100.
The FRC portion 100 includes a motion vector detecting portion 101 that detects motion vector information from the input image signal and an interpolation frame generating portion 102 that generates interpolation frames based on the motion vector information acquired by the motion vector detecting portion 101.
In the above configuration, for example, the motion vector detecting portion 101 may obtain the motion vector information with the use of a block matching method, a gradient method, etc., or if the motion vector information is included in the input image signal in some form, this information may be utilized. For example, the image data compression-encoded with the use of the MPEG format includes motion vector information of a moving image calculated at the time of encoding, and this motion vector information may be acquired.
FIG. 2 is a diagram for explaining frame rate conversion processing by the conventional FRC drive display circuit shown in FIG. 1. The FRC portion 100 generates interpolation frames (gray-colored images in FIG. 2) between frames with the motion compensation processing using the motion vector information output from the motion vector detecting portion 101 and sequentially outputs the generated interpolation signals along with the input frame signals to perform processing for converting the frame rate of the input image signal from 60 frames per second (60 Hz) to 120 frames per second (120 Hz).
FIG. 3 is a diagram for explaining an interpolation frame generation processing of the motion vector detecting portion 101 and the interpolation frame generating portion 102. The motion vector detecting portion 101 uses the gradient method to detect a motion vector 205 from, for example, a frame #1 and a frame #2 shown in FIG. 2. The motion vector detecting portion 101 obtains the motion vector 205 by measuring a direction and an amount of movement in 1/60 of a second between the frame #1 and the frame #2. The interpolation frame generating portion 102 then uses the obtained motion vector 205 to allocate an interpolation vector 206 between the frame #1 and the frame #2. An interpolation frame 207 is generated by moving an object (in this case, an automobile) from a position of the frame #1 to a position after 1/120 of a second based on the interpolation vector 206.
By performing the motion-compensated frame interpolation processing with the use of the motion vector information to increase a display frame frequency in this way, the display state of the LCD (the holding-type display system) can be made closer to the display state of the CRT (the impulse-type display system) and the image quality deterioration may be improved which is due to the motion blur generated when displaying a moving image.
In the motion-compensated frame interpolation processing, it is essential to detect the motion vectors for the motion compensation. For example, the block matching method and the gradient method are proposed as representative techniques for the motion vector detection. In the gradient method, the motion vector is detected for each pixel or small block between two consecutive frames to interpolate each pixel or small block of the interpolation frame between two frames. An image at an arbitrary position between two frames is interpolated at an accurately compensated position to convert the number of frames.
Although the image quality deterioration due to the motion blur caused by the holding-type display may be improved by performing the motion-compensated frame interpolation processing to increase a display frame frequency as above, the input image signal may include motion blurs due to the time integration effect of an image sensor (also called a camera blur), and the image quality is deteriorated by the motion blurs due to the time integration effect of an image sensor. Therefore, for example, a proposal has been made in Japanese Laid-Open Patent Publication No. 2002-373330 for an image processing device removing the motion blurs due to the time integration effect of an image sensor and increasing the feeling resolution without making images unnatural. The conventional image processing device described in Japanese Laid-Open Patent Publication No. 2002-373330 will hereinafter be described with reference to FIG. 4.
FIG. 4 is a functional block diagram of a configuration of a conventional image processing device. An input image supplied to the image processing device is supplied to an object extracting portion 111, an area identifying portion 113, a mixture ratio calculating portion 114, and a foreground/background separating portion 115. The object extracting portion 111 roughly extracts an image object corresponding to a foreground object included in the input image and supplies the extracted image object to a motion detecting portion 112. For example, the object extracting portion 111 roughly extract the image object corresponding to the foreground object by detecting the contour of the image object corresponding to the foreground object included in the input image.
The object extracting portion 111 roughly extracts an image object corresponding to a background object included in the input image and supplies the extracted image object to the motion detecting portion 112. For example, the object extracting portion 111 roughly extracts the image object corresponding to the background object from a difference between the input image and the image object corresponding to the extracted foreground object. For example, the object extracting portion 111 may roughly extract the image object corresponding to the foreground object and the image object corresponding to the background object from a difference between a background image stored in a background memory disposed inside and the input image.
The motion detecting portion 112 uses methods such as a block matching method, a gradient method, a phase correlation method, and a pel-recursive method to calculate a motion vector of the roughly extracted image object corresponding to the foreground object and supplies the calculated motion vector and the position information of the motion vector (information identifying a position of a pixel corresponding to the motion vector) to the area identifying portion 113 and a motion blur removing portion 116. The motion vector output by the motion detecting portion 112 includes information corresponding to a motion amount v. For example, the motion detecting portion 112 may output a motion vector of each of the image objects to the motion blur removing portion 116 along with the pixel position information identifying pixels for the image objects.
The motion amount v is a value representative of a change in a position of an image corresponding to a moving object on the basis of a pixel interval. For example, if an image of an object corresponding to the foreground moves to be displayed at a position four pixels away in the next frame based on a certain frame, the motion amount v of the image of the object corresponding to the foreground is represented by four.
The area identifying portion 113 identifies the respective pixels of the input image as a foreground area, a background area, and a mixture area and supplies information (hereinafter, area information) indicative of which of the foreground area, the background area, or the mixture area each of the pixels belongs to, to the mixture ratio calculating portion 114, the foreground/background separating portion 115, and the motion blur removing portion 116.
The mixture ratio calculating portion 114 calculates a mixture ratio (hereinafter, a mixture ratio α) corresponding to the pixels included in the mixture area based on the input image and the area information supplied from the area identifying portion 113 and supplies the calculated mixture ratio to the foreground/background separating portion 115. The mixture ratio α is a value indicative of a rate of an image component (hereinafter, also referred to as a background component) corresponding to the background objects.
The foreground/background separating portion 115 separates the input image into a foreground component image consisting only of image components (hereinafter, also referred to as foreground components) corresponding to the foreground objects and a background component image consisting only of background components based on the area information supplied from the area identifying portion 113 and the mixture ratio α supplied from the mixture ratio calculating portion 114 and supplies the foreground component image to the motion blur removing portion 116 and the background component image to a correcting portion 117.
The motion blur removing portion 116 determines a unit of processing indicative of one or more pixels included in the foreground component image based on the motion amount v known from the motion vector and the area information. The unit of processing is data that specifies a group of pixels to be processed for adjustment of a motion blur amount. The motion blur removing portion 116 removes the motion blur included in the foreground component image based on the foreground component image supplied from the foreground/background separating portion 115, the motion vector and the position information thereof supplied from the motion detecting portion 112, and the unit of processing and outputs the foreground component image after the removal of the motion blur to a motion blur removed image processing portion 118.
The correcting portion 117 corrects a pixel value of a pixel corresponding to the mixture area in the background component image. A pixel value of a pixel corresponding to the mixture area in the background component image is calculated by removing the foreground component from a pixel value of a pixel of the mixture area before the separation. Therefore, a pixel value of a pixel corresponding to the mixture area in the background component image is reduced correspondingly to the mixture ratio a as compared to a pixel value of a pixel of the adjacent background area. The correcting portion 117 corrects such gain reduction corresponding to the mixture ratio a of a pixel value of a pixel corresponding to the mixture area in the background component image and supplies the corrected background component image to the motion blur removed image processing portion 118.
The motion blur removed image processing portion 118 applies an edge enhancement processing at a different edge enhancement level to each of the foreground component image after the removal of the motion blur and the corrected background component image. For the background component image, which is a still image, the edge enhancement processing is executed to enhance the edges more than the foreground component image. This enables an increase in the feeling resolution of the background component image without generating the unnatural image deterioration at the time of application of the edge enhancement processing to an image including noises.
On the other hand, for the foreground component image, the edge enhancement processing is executed at a lower edge enhancement level as compared to the background component image. This enables the reduction of the unnatural image deterioration while improving the feeling resolution even if the foreground component image after the removal of the motion blur includes noises.